Validating Standards-based Test Score Interpretations

Standards-based score reports interpret test performance with reference to cut scores defining categories like "below basic" or "proficient" or "master." This paper first develops a conceptual framework for validity arguments supporting such interpretations, then presents three applications. Two of these are serve to introduce new standard-setting methods. The conceptual framework lays out the logic of validity arguments in support of standards-based score interpretations, focusing on requirements that the performance standard (i.e., the characterization of examinees who surpass the cut score) be defensible both as a description and as a normative judgment, and that the cut score accurately operationalize that performance standard. The three applications illustrate performance standards that differ in the breadth of the claims they set forth. The first, a "criterion-referenced testing" application, features a narrow performance standard that corresponds closely to performance on the test itself. The second, "minimum competency testing," introduces a new standard-setting method that might be used when there is a weaker linkage between the test and the performance standard. The third, a contemporary standards-based testing application, proposes a new procedure whereby the performance standard would be derived directly from the specification for the test itself. 1 Support for this research from The Spencer Foundation is gratefully acknowledged. The ideas and positions taken are the authors' own; no endorsement by the Spencer Foundation should be inferred. An earlier version of this paper was presented as "Standards for Student Work Versus Standards for Students: Conceptual Confusions in Standards-Based Reporting" in a symposium on "Educational Standards, Assessment, and the Search for 'Consensus'" at the April 2000 annual meeting of the American Educational Research Association. Helpful comments and suggestions by the editors and anonymous reviewers are gratefully acknowledged. Validating Standards-Based Test Score Interpretations